Lifelong Generative Modelling Using Dynamic Expansion Graph Model

Variational Autoencoders (VAEs) suffer from degenerated performance, when learning several successive tasks. This is caused by catastrophic forgetting. In order to address the knowledge loss, VAEs are using either Generative Replay (GR) mechanisms or Expanding Network Architectures (ENA). this paper we study forgetting behaviour of a joint GR and ENA methodology, deriving an upper bound on negative marginal log-likelihood. theoretical analysis provides new insights into how forget previously learnt during lifelong learning. The indicates best performance achieved considering model mixtures, under framework, where there no restrictions number components. However, ENA-based approach may require excessive parameters. motivates us propose novel Dynamic Expansion Graph Model (DEGM). DEGM expands its architecture, according novelty associated with each database, compared information already network previous training optimizes structuring, characterizing probabilistic representations corresponding past more recently learned We demonstrate that guarantees optimal for task while also minimizing required